Skip to content

Evals fixes #2505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
cte merged 2 commits into from
Apr 11, 2025
Merged

Evals fixes #2505

cte merged 2 commits into from
Apr 11, 2025

Conversation

@cte
Copy link
Collaborator

@cte cte commented Apr 11, 2025 •
edited by ellipsis-dev bot
Loading

Context

Implementation

Screenshots

before after

How to Test

Get in Touch

Important

This PR updates database connection handling, adds task duration metrics, and refines UI for model selection and settings import.

  • Database Connection:
    • Updated drizzle.config.ts and db.ts to support both sqlite and turso dialects based on environment variables.
    • Throws error if neither TURSO_CONNECTION_URL nor BENCHMARKS_DB_PATH is set.
  • Task Metrics:
    • Added successfulTaskDurations() in taskMetrics.ts to calculate average, min, and max durations of successful tasks.
  • CLI Changes:
    • In index.ts, socket path now uses a shortened UUID.
    • Git branch name includes a shortened UUID.
  • UI Changes:
    • Removed FormDescription and recommendedModels in new-run.tsx.
    • Adjusted model selection logic to handle different API providers.
    • Improved settings import functionality with error handling and mode switching.

This description was created by Ellipsis for 789cda1. It will automatically update as commits are pushed.

@cte cte requested a review from mrubens as a code owner April 11, 2025 21:53
@changeset-bot
Copy link

changeset-bot bot commented Apr 11, 2025

⚠️ No Changeset found

Latest commit: 789cda1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Apr 11, 2025
@cte cte merged commit 2eba534 into main Apr 11, 2025
15 checks passed
@cte cte deleted the cte/evals-tweaks branch April 11, 2025 21:54
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap Apr 11, 2025
@ellipsis-dev
Copy link
Contributor

ellipsis-dev bot commented Apr 11, 2025

Thank you for your contributions! It looks like this pull request includes changes across different areas of the project, such as CLI refactoring, web app form handling, and database configuration updates. To make the review process more manageable and focused, could you consider splitting this pull request into smaller ones? For example:

  1. CLI Refactoring: Changes related to socket path and git branch naming in evals/apps/cli/src/index.ts.
  2. Web App Form Handling: Updates in evals/apps/web/src/app/runs/new/new-run.ts.
  3. Database Configuration: Changes in evals/packages/db/drizzle.config.ts, evals/packages/db/src/db.ts, and evals/packages/db/src/queries/taskMetrics.ts.

This will help us ensure each part is thoroughly reviewed and tested. Thank you!

SmartManoj pushed a commit to SmartManoj/Raa-Code that referenced this pull request May 6, 2025
@trevhud @saoudrizwan
Trevhud/eng 279 (RooCodeInc#2505)
1efd84a 
* Move restart/delete server and all toggle all

* changeset

* move buttons back to the bottom and show icons even when expanded

* Only display auto-approve all if auto-approve mcp is enabled

* Move auto-approve setting to bottom

---------

Co-authored-by: Saoud Rizwan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mrubens mrubens Awaiting requested review from mrubens mrubens is a code owner

Assignees

No one assigned

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cte